Managing virtual machine instances utilizing a virtual offload device

ABSTRACT

Generally described, the present application relates to systems and methods for the managing virtual machines instances using a physical computing device and an offload device. The offload device can be a separate computing device that includes computing resources (e.g., processor and memory) separate from the computing resources of the physical computing device. The offload device can be connected to the physical computing device via a bus interface. The bus interface can be a high speed, high throughput, low latency interface such as a Peripheral Component Interconnect Express (PCIe) interface. The offload device can be used to offload virtualization and processing of virtual components from the physical computing device, thereby increasing the computing resources available to the virtual machine instances.

BACKGROUND

Generally described, computing devices utilize a communication network,or a series of communication networks, to exchange data. Companies andorganizations operate computer networks that interconnect a number ofcomputing devices to support operations or provide services to thirdparties. The computing systems can be located in a single geographiclocation or located in multiple, distinct geographic locations (e.g.,interconnected via private or public communication networks).Specifically, data centers or data processing centers, herein generallyreferred to as a “data center,” may include a number of interconnectedcomputing systems to provide computing resources to users of the datacenter. The data centers may be private data centers operated on behalfof an organization or public data centers operated on behalf, or for thebenefit of, the general public.

To facilitate increased utilization of data center resources within thedata centers, virtualization technologies may allow a single physicalcomputing device to host one or more instances of virtual machines thatappear and operate as independent computing devices to users of a datacenter. With virtualization, software applications running on thephysical computing device can create, maintain, delete, or otherwisemanage virtual machine instances in a dynamic manner.

Use of the data centers in increasing numbers has created increaseddemand for the computing resources. Even with virtualizationtechnologies, the number of available resources that can be provided tothe virtual machines is limited, at least in part, by the softwareapplications managing the virtual machine instances in the physicalcomputing devices. The cost associated with changing the existinghardware resources for better hardware components can be a considerableexpense.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisdisclosure will become more readily appreciated as the same becomebetter understood by reference to the following detailed description,when taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram depicting a physical computing device with anoffload device and a plurality of virtual machine instances.

FIG. 2A illustrates an embodiment of a state flow diagram depicting theconfiguration of virtual machine instances on the physical computingdevice and virtual components on an offload device utilizing a controlplane manager.

FIG. 2B illustrates an embodiment of a state flow diagram depicting theconfiguration of virtual machine instances on the physical computingdevice and virtual components on an offload device by a virtual machinemonitor.

FIG. 3A illustrates an embodiment of a state flow diagram depictingindirect communication between a virtual machine instance and a virtualcomponent via the virtual machine monitor.

FIG. 3B illustrates an embodiment of a state flow diagram depictingdirect communication between a virtual machine instance and a virtualcomponent.

FIG. 4 illustrates a flow diagram of a routine for determining aconfiguration of a virtual environment on a physical computing deviceand an offload device by a control plane manager.

FIG. 5 illustrates a flow diagram of a routine for the configuration ofa virtual environment on a physical computing device and an offloaddevice.

FIG. 6 illustrates a flow diagram of a routine for indirectcommunication between the host computing device and the virtual devicevia the virtual machine monitor.

FIG. 7 is a block diagram depicting an embodiment of a physicalcomputing device configured with a virtual offload device.

FIG. 8 is a block diagram depicting an embodiment of an environmentincluding a plurality of physical computing configured to communicatewith an offload device.

DETAILED DESCRIPTION

Generally described, a physical computing device can be configured tohost a number of virtual machine instances. Specifically, such physicalcomputing devices can execute a virtual machine monitor can be used tomanage multiple aspects of virtual machine instances. Such a virtualmachine monitor may often be referred to as a “hypervisor.” The virtualmachine monitor can associate and manage three primary virtualizedresources to instantiated virtual machine instances, namely, virtualprocessing resources, virtual memory resources, and virtual input/output(I/O) resources, collectively or individually referred to as virtualcomponents.

Although referred to as virtual components, the instantiation andoperation of the virtual components requires computing resources of thephysical computing device to implement. Generally, the virtual machinemonitor will manage virtual components for each instantiated virtualmachine instance on the physical computing device. As a result, physicalcomputing resources are consumed to support the instantiated virtualcomponents of each instantiated virtual machine instance and reduce theavailability of the physical computing device resources for instantiatedvirtual machine instances or additional virtual machine instances.

The present application relates to systems and methods for the managingthe instantiation and execution of virtual machines instances using aphysical computing device and an offload device. In accordance with anillustrative embodiment, the offload device corresponds to anindependent computing device that includes physical computing resources(e.g., processor and memory) separate from the physical computingresources associated with the physical computing device hosting theinstantiated virtual machine instances. The offload device can beconnected to the physical computing device via an interconnectinterface. The interconnect interface can be a high speed, highthroughput, low latency interface such as a Peripheral ComponentInterconnect Express (PCIe) interface. The offload device can be used tocontrol the virtual machine monitor and emulate certain virtualcomponents associated with the instantiated virtual machine instances,thereby decreasing the need to utilize physical computing resources inthe execution of the instantiated virtual machine instances.

In accordance with an illustrative embodiment, the offload device can beused to instantiate virtual machines on the physical computing device.For example, the offload device can receive a command from a controlplane manager via a network interface integrated into the offload deviceor a management domain and instruct the virtual machine monitor tolaunch virtual machines. In addition, the virtual machine monitor canprovide resource information regarding the physical computing to thecontrol plane manager via the offload device. The control plane managercan determine based on the resource information, such as the specifichardware configuration, and other information, such as the anticipateduse of the physical computing device, the configuration of virtualmachine instances on the physical computing device and virtualcomponents on the offload device. The control plane manager can provideinstructions to the virtual machine monitor to instantiate virtualmachine instances in the determined configuration and instruct theoffload device to instantiate virtual components in the determinedconfiguration. The virtual machine monitor can provide mapping of theinstantiated virtual components on the offload device such that thevirtual machine instances can recognize and communicate with the virtualcomponents through the interface bus.

In accordance with another illustrative embodiment, the virtual machinemonitor can be configured to instantiate virtual machine instances onthe physical computing device and instantiate respective virtualcomponents on the offload device. The configuration of the virtualmachine instances can determine the virtual components that areinstantiated on the offload device on behalf of the instantiated virtualmachine instances. The virtual machine monitor can also provide mappingof the instantiated virtual components on the offload device such thatthe virtual machine instances can recognize and communicate with thevirtual components through the interface bus.

In accordance with an illustrative embodiment, the instantiated virtualI/O components on the offload device are configured to execute orprocess at least a portion of the I/O requests generated by theinstantiated virtual machine instances. Illustratively, the virtualmachine instances can communicate one or more I/O requests with theinstantiated virtual I/O components on the offload device. In someaspects, the instantiated virtual machine instances may communicatedirectly with the virtual I/O components via the interface bus. In otheraspects, the instantiated virtual machine instances can communicateindirectly with the virtual I/O components via the virtual machinemonitor. In this aspect, the virtual machine monitor can use atranslation table to route the I/O request to the virtual component. Thetype of communication can be based, at least in part, on communicationprotocols associated with the virtual I/O components or other criteria.

While specific embodiments and example applications of the presentdisclosure will now be described with reference to the drawings, theseembodiments and example applications are intended to illustrate, and notlimit, the present disclosure. Specifically, while various embodimentsand aspects of the present disclosure will be described with regard toillustrative components of host computing device, one or more aspects ofthe present disclosure can be applied with regard to different types orconfigurations of physical computing devices or combinations thereof.

FIG. 1 illustrates an embodiment of a physical computing device 100configured to host virtual machine instances 120 and interact with anoffload device 130. The physical computing device 100 includes one ormore processing units 102, such as one or more CPUs. The physicalcomputing device 100 includes memory 104, which may correspond to anycombination of volatile or non-volatile computer-readable storage media.The memory 104 may store information which includes various programs,program data, and other modules. The programs stored in the memory caninclude a virtual machine monitor application 110 that can manage thevirtual machine instances (e.g., by allocating memory to each virtualmachine instance and scheduling virtual processors to run on physicalprocessors). The physical computing device 100 can correspond to a widevariety of devices, such as servers, that include a wide variety ofsoftware and hardware components and configurations. The physicalcomputing device 100 can include a local data store (not shown), or beconfigured to communicate with a data store over a network (not shown).

The physical computing device 100 is capable of hosting a plurality ofvirtual machine instances 120. The virtual machine monitor 110 canmanage virtual machine instances 120 hosted on the physical computingdevice 100. Illustratively, the management of the virtual machineinstances 120 can include the instantiation of the virtual machineinstance and the instantiation of virtual components utilized in theexecution of the virtual machine instance. Additionally, as will beexplained in greater detail, the management of the virtual instances canfurther include the management of interaction between the virtualmachine instances and the offload device 130 In the illustratedembodiment, the physical computing device 100 includes two instantiated,or hosted, virtual machine instances 120, virtual machine instance “A”and virtual machine instance “B”. One skilled in the relevant art willappreciate, however, that the physical computing device 100 can host anynumber of virtual machine instances and is not limited to the hosting ofthe two virtual machine instances illustrated in FIG. 1.

With continued reference to FIG. 1, the offload device 130 can beoperably coupled to the physical computing device 100 via theinterconnect interface 106. The interconnect interface 106 can refer toa physical communication interface on the physical computing device 100.The interconnect interface 106 can be an electrical communicationinterface, an optical communication interface or other type ofinterconnect interface known in the art. The interconnect interface 106can be configured to provide communications between components hosted onthe offload device 130 with the virtual machine instances 120 hosted onthe physical computing device 100. Illustratively, the configuration ofthe interconnect interface can be optimized based on specific criteria,such as low latency, high speed, and high bandwidth, among others. Insome embodiments, the interconnect interface can correspond to a highspeed serial computer expansion bus, such as a Peripheral ComponentInterconnect Express (PCIe) bus. However, one skilled in the relevantart will appreciate that the interconnect interface may incorporatealternative or additional standard interconnect interfaces well known tothose skilled in the art of computer architectures.

In an example embodiment, the offload device 130 is a computing device,or partial computing device, operably coupled to the physical computingdevice 100. In some embodiments, the offload device 130 is physicallycoupled to the physical computing device 100 as well. The offload device100 includes one or more processing units 132, such as one or more CPUs.The offload device 130 includes memory 134, which may correspond to anycombination of volatile or non-volatile computer-readable storage media.The memory 134 may store information which includes various programs,program data, and modules, such as a management module 138 and a deviceemulation module 139. The management module 138 can management componentcan be configured to determine the type of virtual components toinstantiate based on configuration information for the virtual machineinstance. The device emulation module 139 can be configured to performthe emulation and instantiation of the virtual components on the offloaddevice 130. The processor 132 and memory 134 of the offload device 130are separate from the processor 102 and memory 104 of the physicalcomputing device 100. The offload device 130 can include a local datastore (not shown), and/or be configured to communicate with a data storeover a network (not shown).

The offload device can include a network interface controller (NIC) 136.The offload device can be in communication with a control plane manager150 (illustrated in FIG. 2A) via a network. The offload device can beconfigured act as an intermediary for providing instructions from thecontrol plane manager 150 to the virtual machine monitor 110, which willbe explained in greater detail below with respect to FIG. 2A.

As will be explained in greater detail below, the offload device 130 canhost and emulate one or more virtual components that are used by theinstantiated virtual machine instances substantially independent of oneor more physical computing device 100 resources. The offload device 130can be dynamically reconfigurable, such that the virtual components 140can be removed, changed, added, or otherwise reconfigured to address theconfiguration of the virtual machine instances 120 on the physicalcomputing device 100. Accordingly, the offload device 130 would use atleast a portion of the physical computing resources on the offloaddevice to carry out the function of the instantiated virtual components.By way of example, operations executed on the offload device 130 can becarried out using the computing resources (e.g., processor 132 andmemory 134) without requiring the usage of the physical computingdevice's 100 computing resources (e.g., processor 102 and memory 104).

In accordance with an illustrative embodiment, at least some portion ofthe virtualized components hosted on the offload device 130 correspondto virtual I/O components configured to execute I/O functionality onbehalf of instantiated virtual machine instances. As illustrated in FIG.1, the offload device 130 can include virtual I/O component groups 142.Each virtual I/O component groups 142 corresponds to a virtual machineinstance 120 on the physical computing device 100. In the illustratedembodiment the offload device 130 includes virtual I/O component group Ais associated with virtual machine instance A and virtual I/O componentgroup B associated with virtual machine instance B. Each virtual I/Ocomponent group includes a plurality of virtual components 140

Generally described, the virtual machine monitor 110 executed on thephysical computing device 100 is configured to manage various aspectsassociated with instantiated virtual machines instances. In anembodiment, the management operations can be split between the virtualmachine monitor and a management domain, such as a Domain-0, that runson physical computing device 100. In yet another embodiment, all orportions of the programs that run within Domain-0 can instead run on theoffload device 130. The virtual machine monitor 110 can be executeddirectly on the physical computing system hardware. The virtual machinemonitor can function like a host operating system for the physicalcomputing device 100. The virtual machine monitor 110 can control thehardware of the physical computing device 100 and manage and configurevirtual machine instances 120 running on the physical computing device100. The virtual machine monitor 110 can implement management functionsthat provide direct access to the hardware of the physical computingdevice 100.

To support hosted virtual machine instances, the virtual machine monitor110 can instantiate guest domains on the physical computing device 100for each virtual machine instances 120 by allocating the guest domainsmemory and time on the physical CPUs. As previously described, theallocated virtual resources include three primary virtualized resourcesthat are utilized by the instantiated virtual machine instances, namely,virtual processing resources, virtual memory resources, and virtual I/Oresources. In some embodiments, the configuration of the virtual machineinstances 120 can be determined by a control plane manager 150 asdescribed in greater detail in FIG. 2A. In some embodiments, the virtualmachine monitor 110 can determine the configuration of the virtualmachine instances as described in greater detail in FIG. 2B.

Each virtual machine instance 120 is provisioned virtual resources thatare implemented by the physical computing resources of the physicalcomputing device 100. For example, a virtual machine instance can beallocated a virtual processing resources 122 and virtual memoryresources 124 that represent logically provisioned allocations ofunderlying computing resources of the physical computing device (e.g.,processor 102 and memory 104). Some of the virtual resources, such asvirtual I/O resources, can be offloaded to the offload device 130. Theconfiguration of the virtualized resources for each virtual machineinstance 120 can vary. For example, virtual machine instance A andvirtual machine instance B can have different allocations of virtualizedcomputing resources.

The virtual machine instances 120 may be provisioned to provide avariety of different desired functionalities depending on the needs of adata center or client. Examples of the types of desired functionalitycan include, but are not limited to: database management, serving ordistributing data or content (e.g., Web servers), managing loadbalancing or network resources, managing network connectivity orsecurity, providing network addressing information, managing client orserver redirection, or other functionalities. In some embodiments, thevirtual machine instances 120 may be provisioned to implement portionsof a hosted network or to simulate one or more components of a hostednetwork. Illustratively, the virtual machine instances 120 may beconfigured to provide specific functionality associated with thecomponents of a hosted network or simulation of the components of thehosted network. The virtual machine instances 120 may be provisionedgenerically when a desired functionality is not specified or isotherwise not available.

As previously describe, aspects of the present application relate to thehosting of virtual I/O components on the offload device 130 in a mannerthat reduces the execution of I/O functionality by the hosted virtualresources on the physical computing device 100. Each virtual machineinstance 120 is associated with virtual components 140 grouped intovirtual I/O component groups 142. The virtual machine monitor 110 isresponsible for the provisioning of virtual I/O component groups 142 foreach of the virtual machine instances 120. The virtual components 140can be logically grouped based on their association with a virtualmachine instance 120. The virtual machine monitor 110 can assign memoryaddress ranges to virtual components within the memory allocated tovirtual machine instances. For example, in FIG. 1, virtual I/O componentgroup A is associated with virtual machine A and the virtual I/Ocomponent group B is associated with virtual machine B. In someembodiments, the virtual components 140 can be provisioned and emulatedusing the computing resources (e.g., processor 132 and memory 134) ofthe offload device 130. For example, the offload device 130 can run oneor more programs that simulate the functions of hardware components thatwould be typically found on a motherboard of a computer system.

The virtual components 140 represent a set of virtual functions that canbe implemented by a virtual machine instances 120. The virtualcomponents 140 can provide virtual I/O functions that emulate the I/Ofunctions of hardware computing devices found in a physical computingdevice. For example, the virtual components can correspond to I/O devicetypes such as the real time clock (RTC), storage controller, networkinterface controller (NIC), programmable interrupt controller (PIC),peripheral component interconnect (PCI) bus, disk controller, SCSIcontroller, floppy drive, keyboard and mouse ports, monitor ports,serial ports, keyboard controller, ISA bus, and other I/O devices. Thevirtual components 140 are sometimes referred to as virtual devices. Ina virtual computing environment not every function needs to bevirtualized for every machine. The virtual machine monitor 110 candetermine which I/O devices need to be virtualized based on theconfiguration of the virtual machine instance 120.

In addition to the functionality implemented by the virtual I/Ocomponents, the various virtual components 140 can be configured as tothe specific communication protocols utilized by the instantiatedvirtual machines to access the I/O functionality. More specifically, inone embodiment, some of the virtual I/O components may correspond to aPort I/O communication protocol (“Port I/O virtual components”) andother virtual I/O components may correspond to a memory-managed I/O(MMIO) communication protocol (“MMIO virtual components”).

Port I/O virtual components can require specific communication protocolsfor communications between a virtual machine instance 120 and a Port I/Ovirtual component. In this embodiment, the virtual machine monitor 110can function as an intermediary to handle the communication protocolsrequired for the communication between the Port I/O virtual componentsand the virtual machine instance 120. The virtual machine monitor 110can include a translation table that is used for handling communicationbetween the virtual machine instance and port I/O virtual components140, some MMIO components also use a translation table. The translationtable can include a table for translating requests to and from thevirtual machine instance 120 in order to route the requests to thecorrect addresses assigned to the virtual components 140. Additionaldetails regarding the communication between virtual machine instances120 and virtual components 140 that utilize the virtual machine monitor110 are described below in associated with FIG. 3A.

The MMIO components can be configured such that the virtual machineinstance 120 can communicate directly with the virtual component 140 bycommunicating with the memory addresses assigned to the virtualcomponent 140. Additional details regarding direct communication betweenvirtual machine instances 120 and virtual components 140 described belowin associated with FIG. 3B.

The physical computing device 100 can be part of a network that includesmultiple physical computing devices 100. One skilled in the relevant artwill appreciate that the network is logical in nature and can encompassphysical computing devices 100 from various geographic regions.Additionally, the network can include one or more physical computingdevices 100 that do not host virtual machine instances 120.

The physical computing devices can be managed by a centralizedmanagement system, such as illustrated by the control plane manager 150in FIG. 2A. The control plane manager 150 can be configured to managethe operation and configuration of the physical computing devices 100 onthe virtual network as well as select computer systems to host virtualmachines and send launch instructions to the offload device 130 or amanager program that runs in Domain-0. The control plane manager 150 candetermine configurations, operating parameters, resource allocationswithin virtual machine instances, for each physical computing devicewithin a virtual network. In some embodiments, the managements systemcan comprise a plurality of control plane managers that controldifferent allocations of physical computing devices. The control planemanager 150 can be in communication with the physical computing devices100 through the offload device 130. In some embodiments, the controlplane managers can communicate directly with the offload device 130and/or the physical computing devices 100.

FIG. 2A illustrates a block diagram depicting the configuration ofvirtual machine instances 120 on the physical computing device 100 andvirtual components 140 on the offload device 130 by a control planemanager 150. The control manager 150 can be in communication with thephysical computing device 100 via the offload device 130. The functionsdescribed in association with FIG. 2A can be performed in a differentsequence, can be added, merged, or left out altogether (e.g., not alldescribed acts or events are necessary for the practice of thealgorithms). Moreover, in certain embodiments, functions, acts or eventscan be performed concurrently. For example, the operations associatedwith (6), (7) and (8) can occur in a different order or can occurconcurrently.

At (1), the virtual machine monitor 110 can determine the availableresources of the physical computing device. The available resources caninclude information such as the available computing resources, such asprocessor type, memory configuration, hardware component configuration,versioning information, and other information that identifies theresources of the physical computing device. The resource information caninclude current operating conditions, computer resource utilizationinformation associated with the current configuration of the physicalcomputing device 100. In some embodiments, the resource information caninclude resource information associated with the offload device 130. Theresource information can be gathered based on a request provided by theoffload device or the control plane manager. In some embodiments, thevirtual machine monitor may be configured to provide the resourceinformation based on a triggering condition, such as during a portion ofthe boot process. In some embodiments, the virtual machine monitor 110can be configured to periodically transmit the resource information.

In some embodiments, the resource information associated with a physicalcomputing device 100 can be stored in a data store configured formaintaining the resource information associated with each physicalcomputing device. The data store can also store the current operationalstate of the physical computing device, such as the instances that arerunning on the physical computing device 100.

At (2) the virtual machine monitor sends the resource information to theoffload device. After receipt of the resource information, at (3) theoffload device can send the resource information to the control planemanager. In some embodiments, the resource information provided by thevirtual machine monitor 110 can be supplemented with resourceinformation associated with the offload device 130. In such instances,the resource information may be sent together with the virtual machinemonitor resource information or sent separately. In some embodiments,the offload device 130 can periodically send information to the controlplane manager 150 to provide an update regarding the status of instancesthat are operating on the physical computing device 100.

At (4), the control plane manager can determine the configuration of thephysical computing device 100 and the offload device 130. Thedetermination can be based, in part, on the received resourceinformation and information independent of the resource information. Forexample, the control plane manager 150 can also base the configurationon other considerations, such as client specifications, theconfigurations of other physical computing devices 100, such asclustered computing devices, or other considerations independent of theresource information associated with the physical computing device. Insome instances the control plane manager can be configuring a newphysical computing device 100, updating an existing configuration aphysical computing device 100, adding/removing virtual instances, and/orperforming other management function associated with the physicalcomputing device 100.

The control plane manager 150 can determine the configuration of theallocation of the virtual machine instances on the physical computingdevices and virtual components on the offload devices. As part of theconfiguration of the physical computing device 100, the control planemanager 150 can determine the virtualized hardware resources allocatedto each virtual machine instance 120. A virtual machine instance 120 canhave a specific configuration according to the computing capacity of thevirtual machine instance. This can be based on the resource information,requirements of a customer, the system, number of instances operating ona physical computing device 100, and other considerations.

Based on the specific configuration of the virtual machine instances120, the control plane manager 150 can determine the virtual components140 that are associated with a virtual machine instance. The virtualmachine instances 120 may have different specifications associated withthe software and/or hardware of the virtual machine instance 120. Thedifferent specifications for the virtual machine instance 120 mayrequire specific virtual components 140, which may differ from thevirtual components configured for other virtual machine instances 120.

In some instances, the control plane manager 150 can receive a requestto launch a new instance. Based on the request, the control planemanager 150 can filter out physical computing devices 100 that cannothost the instance, such as physical computing devices that are full,physical computing devices that do not have the necessary hardware,physical computing devices that are already hosting too many instances,or physical computing devices that do not meet the requirements for thenew instance based on other reasons. The control plane manager canselect a physical computing device from the remaining physical computingdevices and sends a launch command to the offload device 130 or physicalcomputing device with configuration instructions for launching therequested instance having a specific configuration.

At (5), configuration instructions are sent to the offload device 130for configuration of the virtual components on the offload device 130.In some embodiments, the configuration instructions for the virtualmachine monitor are included with the offload device configurationinstructions. In addition, in some embodiments the configurationinstructions can be sent to the offload device 130 or a manager runningin Domain0 as part of an instance launch command. In this example, thecontrol plan manager 150 may have selected the physical computing device100 to host a virtual machine and sent a command to launch an instanceto the physical computing device 100.

At (6) the offload device can instantiate one or more virtual components140 on the offload device 130 based on the configuration instructionsreceived from the control plane manager. The virtual components 140 onthe offload device 130 represent virtual IO functions that areprovisioned for use by a virtual machine instance 120. Some non-limitingexamples of virtual components 140 that may be instantiated on anoffload device 130 include, a network interface controller (NIC), aprogrammable interrupt controller (PIC), a keyboard controller, an ISAbus, a floppy drive, a keyboard port, a mouse port, a monitor port, anda serial port. Some of the virtual components 140 instantiated for thevirtual machine instance 120 can be based on the configuration of thevirtual machine instance 120. Some of the virtual components 140 may berequired for operation of the virtual machine instance 120, regardlessof whether the virtual component 140 will be used. Some of the virtualcomponents 140 may be virtualizations of hardware components that existon the offload device, whereas others may be emulations of hardwarecomponents that do not exist on the offload device.

At (7), configuration instructions are sent to the physical computingdevice from the offload device 130 for configuration of the virtualmachine instances on the physical computing device 100. In someembodiments, the configuration instructions for the virtual machinemonitor 110 are sent directly from the control plane manager 150.

At (8), the virtual machine monitor 110 can instantiate the virtualmachine instances 120 based on the configuration instructions providedby the control plane manager 150 via the offload device 130. In someembodiments, the control plane manager 150 can communicate directly withthe virtual machine monitor 110. The virtual machine monitor 110provisions the logical virtualized resources that are associated withthe underlying physical resources to each virtual machine instance, suchas a VM processor 102 and VM memory 104 based on the configurationinstructions. The virtual machine monitor 110 can also provision storageresources that are included locally on the physical computing device 100or that are accessible via network.

In some embodiments, the virtual I/O components can be instantiated onthe physical computing device 100 and the offload device 130. In suchembodiments, a portion of the virtual I/O components are instantiated onthe physical computing device 100 and a portion of the virtual I/Ocomponents are instantiated on the offload device 130. The division ofthe virtual I/O components between the physical computing device 100 andthe offload device 130 can be determined by the configurationinstructions.

In some embodiments, the virtual machine monitor can include a memorymapping unit that manages the mapping of virtual components 140instantiated on the offload device 130 to the virtual machine instance120. The virtual machine monitor 110 can assign the virtual components140 to memory addresses of the offload device. The addresses mapped toeach virtual component 140 are provided to the virtual machine instance120 associated with virtual component 140. The virtual components 140associated with same virtual machine instance 120 may not besequentially arranged within the memory of the offload device. In someembodiments, the virtual machine monitor 110 can assign ranges of memoryaddresses on the offload device to each virtual machine instance 120.For example, if there were 12 virtual machine instances 120 the virtualmachine monitor 110 could assign separate ranges of memory addresses ofthe offload device 130 to each of the 12 virtual machine instances 120.

In other embodiments, the hosted virtual components 140 can beconfigured to communicate directly with a virtual machine instance 120.These virtual components 140 can be MMIO virtual components 140. In thisinstance the offload devices exposes or otherwise provides access to thevirtual components 140 directly to the virtual machine instance 120.Some virtual components 140 can be configured to communicate indirectlywith the virtual machine instance 120. For indirect communication, thevirtual machine instance 120 communicates with the virtual component viathe virtual machine monitor 110. The virtual machine monitor can createa translation table that is used to direct the IO communications betweenthe virtual machine instance 120 and the offload device 130. Differenttranslation tables can be created for port I/O virtual components andMMIO virtual components. The processes for direct and indirectcommunication between the virtual machine instances 120 and the virtualcomponents 140 are described in more detail with respect to FIGS. 3A and3B.

FIG. 2B illustrates a block diagram depicting the configuration ofvirtual machine instances 120 on the physical computing device 100 andvirtual components 140 on the offload device 130 by the virtual machinemonitor 110. The functions described in association with FIG. 2A can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments,functions, acts or events can be performed concurrently.

At (1), the virtual machine monitor 110 can configure the physicalcomputing device 100 and the offload device 130. The virtual machinemonitor 110 runs on the hardware of the physical computing device 100and can communicate with the offload device 130. The virtual machinemonitor 110 can configure guest domains for each of the virtual machineinstances 120 on the physical computing device 100 and configure accessto virtual components 140 on the offload device 130.

The virtual machine monitor can configure any number of virtual machineinstances 120. The virtual machine instances 120 can be configuredautomatically based on defined criteria, instructions from othersystems, or other types of instructions. The virtual machine monitor canalso provide a management interface that can allow an administrator orother user to manually configure the virtual machine instances 120 onthe physical computing device 100.

With continued reference to FIG. 2B, as part of the configuration of thephysical computing device 100, the virtual machine monitor 110 candetermine the virtualized hardware resources allocated to each virtualmachine instance 120. A virtual machine instance 120 can have a specificconfiguration according to the computing capacity of the virtual machineinstance. This can be based on the requirements of a customer, thesystem, number of instances operating on a physical computing device100, and other considerations. The configuration information can includeconfiguring a translation table that is used for memory mapping to theI/O devices.

Based on the specific configuration of the virtual machine instances120, the virtual machine monitor 110 can determine the virtualcomponents 140 that are associated with a virtual machine instance. Thevirtual machine instances 120 may have different specificationsassociated with the software and/or hardware of the virtual machineinstance 120. The different specifications for the virtual machineinstance 120 may require specific virtual components 140, which maydiffer from the virtual components configured for other virtual machineinstances 120.

After the configuration of a virtual machine instance 120 is determined,at (2), the virtual machine monitor 110 can instantiate the virtualmachine instances 120. The virtual machine instances 120 provide adomain for operating systems and applications to run. The virtualmachine instances can be fully isolated from other virtual machineinstances 120. The virtual machine monitor 110 provisions the logicalvirtualized resources that are associated with the underlying physicalresources to each virtual machine instance, such as a VM processor 102and VM memory 104. The virtual machine monitor 110 can also provisionstorage resources that are included locally on the physical computingdevice 100, on the offload device 130, or that are accessible vianetwork. In combination with the provisioning of the virtual machineinstances 120, the virtual machine monitor is also responsible forprovisioning the IO virtual components 140 that are required by thevirtual machine instance.

In some embodiments, the virtual I/O components can be instantiated onthe physical computing device 100 and the offload device 130. In suchembodiments, the virtual machine monitor 110 instantiates a portion ofthe virtual I/O components on the physical computing device 100 and aportion of the virtual I/O components on the offload device 130. Thedivision of the virtual I/O components between the physical computingdevice 100 and the offload device 130 can be determined by theconfiguration data associated with the virtual machine instance 120.

At (3) the virtual machine monitor 110 or a Domain-0 can cause theoffload device 130 to instantiate one or more virtual components 140 bysending information that identifies the type and umber of virtualcomponents to instantiate. The virtual components 140 on the offloaddevice 130 emulate functions performed by I/O physical components. Somenon-limiting examples of virtual components 140 that may be instantiatedon an offload device 130 include, a storage device, a network interfacecontroller (NIC), a programmable interrupt controller (PIC), a keyboardcontroller, an ISA bus, a floppy drive, a keyboard port, a mouse port, amonitor port, and a serial port. Some of the virtual components 140instantiated for the virtual machine instance 120 can be based on theconfiguration of the virtual machine instance 120. Some of the virtualcomponents 140 may be required for operation of the virtual machineinstance 120, regardless of whether the virtual component 140 will beused. Some of the virtual components 140 may be virtualizations ofhardware components that exist on the offload device, whereas others maybe virtualizations of hardware components that do not exist on theoffload device.

In some embodiments, the virtual machine monitor can include a memorymapping unit that manages the mapping of virtual components 140instantiated on the offload device 130 to the virtual machine instance120. The virtual machine monitor 110 can assign the virtual components140 to memory addresses of the offload device. The addresses mapped toeach virtual component 140 are provided to the virtual machine instance120 associated with virtual component 140. The virtual components 140associated with same virtual machine instance 120 may not besequentially arranged within the memory of the offload device. In someembodiments, the virtual machine monitor 110 can assign ranges of memoryaddresses on the offload device to each virtual machine instance 120.For example, if there were 12 virtual machine instances 120 the virtualmachine monitor 110 could assign separate ranges of memory addresses ofthe offload device 130 to each of the 12 virtual machine instances 120.

In other embodiments, the hosted virtual components 140 can beconfigured to communicate directly with a virtual machine instance 120.These virtual components 140 can be MMIO virtual components 140. In thisinstance the offload devices exposes or otherwise provides access to thevirtual components 140 directly to the virtual machine instance 120.Some virtual components 140 can be configured to communicate indirectlywith the virtual machine instance 120. For indirect communication, thevirtual machine instance 120 communicates with the virtual component viathe virtual machine monitor 110. The virtual machine monitor can createa translation table that is used to direct the IO communications betweenthe virtual machine instance 120 and the offload device 130. Differenttranslation tables can be created for port I/O virtual components andMMIO virtual components. The processes for direct and indirectcommunication between the virtual machine instances 120 and the virtualcomponents 140 are described in more detail with respect to FIGS. 3A and3B.

FIG. 3A illustrates the components of the virtual network depictingcommunication between a virtual machine instance 120 and a virtualcomponent 140 that uses the virtual machine monitor 110 as anintermediary between the virtual machine instance 120 and the virtualcomponent. The diagram illustrates indirect communication between thevirtual machine instance 120 and virtual component 140. Indirectcommunication can occur when a virtual component 140 virtualizes a PortI/O device and some MMIO devices. The devices can require communicationprotocols that do not allow the virtual machine instance 120 to directlyaccess the assigned memory addresses of the virtual components 140.

At (1) an I/O request from the virtual machine instance 120 triggers avirtual machine exit, also referred to as a VM exit. A VM exit is inresponse to certain I/O requests and/or events that marks the point atwhich a transition is made between the virtual machine instance 120currently running and the virtual machine monitor 110, which mustexercise system control to process the request. Virtual components 140that use indirect communication, and thus trigger VM exits, aredetermined during initialization and instantiation of the virtualmachine instance 120 and virtual components 140. The I/O requests thatcan trigger the VM exit are Port I/O requests and some identified MMIOrequests. The I/O request identifies a specific virtual function of avirtual component 140 and includes instructions for the identifiedvirtual component 140. When the VM exit is triggered by the I/O request,the virtual machine instance 120 sends the I/O request to the virtualmachine monitor 110.

At (2) the I/O request is received and translated by the virtual machinemonitor 110. The virtual machine monitor 110 receives the request anduses a translation table for translating the I/O request. Thetranslation table can include entries for each virtual component 140that requires a VM exit. The virtual machine monitor can includeseparate translation tables for different types of virtual components140. For example, Port I/O virtual components and MMIO virtualcomponents can have different translation tables. In some embodiments,the translation table can combine the translation information for PortIO virtual components 140 and MMIO virtual components 140 into a singletable. The configuration of the translation table can be preconfiguredby the virtual machine monitor. The translation table can store therouting information used for routing the received I/O request to theidentified virtual component 140. After the I/O request is received, thevirtual machine monitor can look up the I/O request in the translationtable and route the request to the memory address of the offload device130 that is responsive to the I/O request.

At (3), the I/O request is routed to the identified virtual component140B. The I/O request is sent from the physical computing device 100over the interface to the offload device 130. At (4) the I/O request isreceived and resolved by the virtual component 140B. The virtualcomponent 140B can resolve the request based on the informationcontained in the I/O request. The virtual component 140 can resolve therequest based on the virtual function that is assigned to the memoryaddress identified in the I/O request. The processing of the request isperformed by the computing resources of the offload device and does notutilize the computing resources assigned to the virtual machine instance120. The I/O request can be a simple read or write, or a complexalgorithm that is implemented on the offload device. For example, theoffload device 130 may execute one or more device emulator programs. Theoffload device 130 may process the request and identify the appropriateemulator based on characteristics of the request. Next, the deviceemulator can run and process the request. For virtual components 140 setup on the offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

Based on the type of the request, the resolution of the requestperformed by the virtual component 140 can differ. In some instances,the I/O request may require a response from the virtual component 140B.In some instances, the I/O request is resolved and may send anacknowledgment back to the virtual machine instance. For example, awrite command from the virtual machine instance 120 to the virtualdevice may require a response from the virtual component 140 to thevirtual machine instance 120. In which case, an acknowledgment can besent from the virtual component 140 to the virtual machine instance 120.

In some instances, the process continues when an I/O request requires aresponse from the virtual component 140 to the virtual machine instance,such as a read request. In which case, after the request is resolved thevirtual component 140 responds to the request. At (5), the virtualcomponent 140 sends a response to the I/O request. The response caninclude the information specific to the request and is dependent on thespecific parameters of the I/O request and the virtual component, or maybe an acknowledgement.

At (6), the virtual machine monitor 110 translates that response to theI/O request. The translation of the response is performed using thetranslation table to route the response to the virtual machine instance120 that initiated the request. At (7), the response to the I/O requestis sent to the identified virtual machine instance 120 from the virtualmachine monitor based on the information stored in the translation tableof virtual machine monitor. The response can be accompanied by a VMresume that closes out the VM exit. At this point, the process completesand can be reinitiated each time an IO request is required that uses thetranslation table of the virtual machine monitor to act as anintermediary between the virtual machine instance 120 and the virtualcomponent.

FIG. 3B illustrates a block diagram depicting direct communicationbetween a virtual machine instance 120 and a virtual component. Virtualmachine instances 120 can communicate directly with the MMIO virtualcomponents. The MMIO virtual components 140 allow for directcommunication between the virtual machine instance 120 and the memoryregisters assigned to the virtual component 140 without any interactionbetween the virtual machine instance 120 and the virtual machine monitor110.

At (1) the virtual machine transmits an I/O request to a virtualcomponent. The virtual machine instance 120 transfers the request to thevirtual component 140 using memory addressing assigned to the virtualcomponent 140 during instantiation of the virtual machine instance 120and the virtual component. The virtual component 140 can be assigned toa range of memory addresses for communication with the virtual machineinstance. Depending on the type of request the virtual machine instance120 can communicate with a specific memory register. The memory mappingof the virtual component 140 allows the virtual machine instance 120 tocommunicate directly with the virtual component 140 through theinterconnect interface 106. A memory management unit can translatedevice-visible virtual memory addresses to physical memory addresses onthe offload device 130.

At (2), the I/O request is resolved by the device. The I/O request canbe any type of request such as a read or write request. The virtualcomponent 140 can resolve the request based on the specific parametersof the request. The processing of the request is performed by thecomputing resources of the offload device and does not utilize thecomputing resources assigned to the virtual machine instance 120. TheI/O request can be a simple read or write, or a complex algorithm thatis implemented on the offload device. For virtual components 140 set upon the offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

At (3), the virtual component 140 can generate a response and send it tothe virtual machine instance. For example, an acknowledgment can be sentfrom the virtual component 140 to the virtual machine instance 120.

FIG. 4 illustrates a flow diagram of a routine 400 depicting theconfiguration of a virtual environment on a physical computing device100 and an offload device 130. The steps of the routine 400 are beingdescribed as generally being performed by a control plane manager 150.The functions described in association with FIG. 4 can be performed in adifferent sequence, can be added, merged, or left out altogether (e.g.,not all described acts or events are necessary for the practice of thealgorithms). Moreover, in certain embodiments, functions, acts or eventscan be performed concurrently.

At block 402, the control plane manager 150 receives resourceinformation associated with the physical computing device and theoffload device. The resource information can include information such asthe available computing resources, such as processor type, memoryconfiguration, hardware component configuration, versioning information,and other information that identifies the resources of the physicalcomputing device and the offload device. The resource information caninclude current operating conditions, computer resource utilizationinformation associated with the current configuration of the physicalcomputing device 100 and the offload device. The resource informationcan be gathered based on a request provided by the offload device 130 orthe control plane manager 150.

As part of a request to launch a virtual machine, and as illustrated atblock 404, the control plane manager can determine a configuration for avirtual machine instance to launch on the physical computing device 100.The determination can be based, in part, on the resource information andinformation independent of the resource information. For example, thecontrol plane manager 150 can also base the configuration on otherconsiderations, such as client specifications, the configurations ofother physical computing devices 100, such as clustered computingdevices, or other considerations independent of the resource informationassociated with the physical computing device.

As part of determining the configuration of the virtual machine, thecontrol plane manager 150 can determine the virtualized hardwareresources that will need to be allocated to the virtual machine instance120. A virtual machine instance 120 can have a specific configurationaccording to the computing capacity of the virtual machine instance.This can be based on the resource information, requirements of acustomer, the system, the number of instances operating on the physicalcomputing device 100, and other considerations. The virtual machineinstances 120 may have different specifications associated with thesoftware and/or hardware of the virtual machine instance 120. Thedifferent specifications for the virtual machine instance 120 mayrequire specific virtual components 140, which may differ from thevirtual components configured for other virtual machine instances 120.

At block 406, the configuration instructions are provided to the offloaddevice 130 for configuration of the virtual components on the offloaddevice 130. The offload device can instantiate one or more virtualcomponents 140 on the offload device 130 based on the configurationinstructions received from the control plane manager. Based on thespecific configuration of the virtual machine instances 120, the offloaddevice 130 and/or the virtual machine monitor can determine the virtualcomponents 140 to instantiate on the offload device 130.

At block 408, configuration instructions are provided to the physicalcomputing device from the offload device 130 for configuration of thevirtual machine instances on the physical computing device 100. In someembodiments, the configuration instructions for the virtual machinemonitor 110 are sent directly from the control plane manager 150 to thevirtual machine monitor. The virtual machine monitor 110 can instantiatethe virtual machine instances 120 based on the configurationinstructions provided by the control plane manager via the offloaddevice. In some embodiments, functions associated with blocks 406 and408 can occur substantially simultaneously in response to a instancelaunch request.

FIG. 5 illustrates a flow diagram of a routine 500 depicting theconfiguration of a virtual environment on a physical computing device100 and an offload device 130. The steps of the routine 500 are beingdescribed as generally being performed by a virtual machine monitor 110on the physical computing device 100 and/or by management software onthe offload device 130, such as the management module 138 and deviceemulation module 139.

At block 502 the virtual machine monitor 110 and management softwarerunning on offload device 130 boot on the physical computing device 100.The virtual machine monitor 110 can instantiate instances on thephysical computing device 100. The virtual machine monitor 110 cancontrol the provisioning of resources of the hardware resources from themachine to specific virtual machine instances 120 and those virtualmachine instances 120 can then be logically associated with theunderlying hardware resources.

After a request to launch an instance is received, and as illustrated atblock 504, the virtual machine monitor 110 can determine theconfiguration of the virtual machine instances 120. The determination ofthe virtual machine instances 120 can be based on information providedfrom an administrator or control plane manager 150. The configurationcan be a default configuration based on defined operating proceduresstored within the virtual machine monitor 110 or it can be passed to theoffload device 130 from the control plane manager 150. In some instancesthe virtual machine monitor may be manually controlled by a user inorder to determine the configuration for the virtual machine instances120. The determination of the configuration of the virtual machineinstances 120 can include the number of virtual machine instances 120,the resources provisioned to each virtual machine instance, one or moremachine image to use to generate each virtual machine instance, andother virtual machine instance 120 configuration information. The numberof virtual machine instances 120 can be based on the computing resourcesof the physical computing device 100, the offload device 130, or otherconfiguration parameter. The number of virtual machine instances 120 maynot limited by any predefined limit. The provisioning of virtualcomputing resources assigned to each virtual machine instance 120 can bebased on logical physical resources of the physical computing device100. The specific configuration of the virtualized computing resourcescan vary for each virtual machine instance. The virtualized computingresources can include a virtual machine CPU, virtual machine memory andother resources used for the operation of the virtual machine instance.The machine image used to instantiate a virtual machine instance caninclude the operating system (e.g., Windows®, Linux®, etc.),applications, and any other software. The type of virtual components 140required by the virtual machine instance 120 can be based on thespecific configuration of the virtual machine instance. The virtualmachine monitor 110 can determine the allocation of virtual components140 between the physical computing device 100 and the offload device130. The virtual machine instance 120 may be configured with somevirtual components 140, such as data stores, and the offload device 130may be configured with some virtual components 140. In some embodiments,the virtual components 140 may be allocated primarily to the offloaddevice 130.

At block 506, the virtual machine monitor can instantiate the virtualmachine instances 120 on the physical computing device 100. Each virtualmachine instance 120 is configured within a guest domain on the physicalcomputing device 100. Each guest domain is configured to be independentof the other guest domains. The instantiation of the virtual machineinstance 120 includes the determined configuration of computingresources, software, and other components as determined at block 404. Inan example embodiment, the virtual machine monitor can determine theallocation of the virtual components 140 between the physical computingdevice 100 and the offload device 130. For example, the virtual machinemonitor may configure the physical computing device 100 to not have anyvirtual components 140 allocated to the physical computing device 100and all of the virtual components 140 allocated to the offload device130. Alternatively, the management module 138 on the offload device 130can determine the allocation of virtual components 140 between thephysical computing device 100 and the offload device 130.

At block 508, the virtual components 140 can be configured on theoffload device 130. For example, either the virtual machine monitor, adomain-0 management program, or management programs on the offloaddevice 130, such as the management module 138 and the device emulationmodule 139, can configure the virtual components 140. The number andtype of virtual components 140 on the offload device 130 can be based onthe specific configuration of the virtual machine instance. The virtualcomponents 140 can include MMIO virtual components 140 and Port IOvirtual components 140. The virtual components 140 on the offload device130 can be logically partitioned according to their associated virtualmachine instance 120. The virtual components 140 are associated withvirtual functions. The virtual components 140 are instantiated in thememory of the offload device 130. In some instances the offload device130 can have defined partitions that that include sequential ranges ofmemory assigned to a virtual machine instance. In some embodiments, thevirtual components 140 on the offload device 130 are assigned to logicallocations within the memory, which may not be sequential in nature. Theinstantiation of the virtual components 140 on the offload device 130allows for the physical computing device 100 to not have to allocatememory to the instantiation of the virtual components 140. Some of thevirtual components 140 may never be using in a virtual operatingenvironment but are required for the operation of the virtual machineinstance. The instantiation of the virtual components 140 can result inresource usage overhead that reduces the available computing resourcesthat are provisioned to the virtual machine instances 120. By allocatingthe virtual components 140 on the offload device 130, computingresources on the physical computing device 100 can be freed up for usageby the virtual machine instances 120.

At block 510, the virtual machine monitor can determine the memorymapping of the virtual components 140 and the communication between thevirtual components 140 and the virtual machine instance. A memorymanager unit can determine the memory mapping for enabling communicationbetween the virtual machine instances and the virtual components 140.Depending on the type of virtual component, the virtual machine monitorcan allow for direct access to the virtual component 140 or can provideindirect access via the virtual machine monitor. For direct access thevirtual machine monitor assigns addresses to the virtual components 140that allow the virtual machine instance 120 to communicate directly withthe virtual components. For indirect access, the virtual machine monitor110 can configure a translation table for communication between thevirtual machine instances 120 and the virtual components 140. Thetranslation table can include addressing information for port I/Ovirtual components 140 and some MMIO virtual components 140.

The instantiation of the virtual components 140 on the offload device130 can be done in parallel with the instantiation of the virtualmachine instance 120 on the physical computing device 100. Some stepscan be done in parallel while others may be done sequentially. The stepsare merely illustrative of logical processes that are being performed bythe virtual machine monitor on the physical computing device 100 and theoffload device 130.

FIG. 6 is a flow diagram illustrating a routine 600 depicting indirectcommunication between the host computing device and the virtual devicevia the virtual machine monitor 110. The steps of the routine 600 arebeing described as generally being performed by a virtual machinemonitor 110 on the physical computing device 100.

At block 602 the virtual machine monitor receives an I/O request from avirtual machine instance 120, which is triggered by a VM exit. When theVM exit is triggered by the I/O request, the virtual machine instance120 sends the I/O request to the virtual machine monitor. The I/Orequest comes from the virtual machine instance 120 for a specificvirtual component.

At block 604, the I/O request is translated by the virtual machinemonitor 110. The virtual machine monitor receives the request and has atranslation table that is used for translating the I/O request. Thetranslation table can include entries for each virtual component 140that requires a VM exit. The virtual machine monitor can includeseparate translation tables for different types of I/O devices. Forexample, Port IO virtual components 140 and MMIO virtual components 140can have different translation tables. In some embodiments, thetranslation table can combine the translation information for Port IOvirtual components 140 and MMIO virtual components 140. Theconfiguration of the translation table can be preconfigured by thevirtual machine monitor. The translation table can be a lookup tablethat can store the routing information for the virtual components 140.When the I/O request is received, the virtual machine monitor can lookup the I/O request in the translation table and determine the routinginformation for directing the I/O request to the correct locationassociated with the addressed virtual component on the offload device130.

At block 606, the virtual machine monitor 110 routes the I/O request tothe identified virtual component. The I/O request is sent from thephysical computing device 100 over the interface bus to the offloaddevice 130. The processing of the request is performed by the computingresources of the offload device and does not utilize the computingresources assigned to the virtual machine instance 120. The I/O requestcan be a simple read or write, or a complex algorithm that isimplemented on the offload device. For virtual components 140 set up onthe offload device, the type of request is not considered whendetermining whether route the request using the offload device. Rather,the virtual machine monitor 110 routes the request regardless of theoperations that are to be performed based on the request.

At block 608, the virtual machine monitor receives a response to the I/Orequest from the virtual component. When a response is not required, thevirtual machine monitor 110 can receive an acknowledgment from thevirtual component 140. The response can include the informationresponsive to the request, which can be dependent on the specificparameters of the IO request and the virtual component.

At block 610, the virtual machine monitor 110 translates that responseto the I/O request. The translation of the response is performed usingthe translation table to route the response to the virtual machineinstance 120 that initiated the request.

At block 612, the response to the I/O request is sent to the identifiedvirtual machine instance 120 from the virtual machine monitor based onthe information stored in the translation table of virtual machinemonitor. The response can be accompanied by a VM resume that closes outthe VM exit. At this point the process completes and can be reinitiatedeach time an IO request is required that uses the translation table ofthe virtual machine monitor to act as an intermediary between thevirtual machine instance 120 and the virtual component.

FIG. 7 illustrates another embodiment of a virtual operating environmentfor a physical computing device 100 with a virtual offload device 230.The physical computing device 100 does not have a physical offloaddevice like the embodiment illustrated in FIG. 1. In this embodiment ofthe virtual offload device 230, the physical computing system hosts aplurality of virtual machine instances 120 and a virtual offload device230. The virtual offload device 230 is partitioned to a specific portionof the physical computing device 100. The virtual offload device 230 isinstantiated using the computing resources from the physical computingdevice 100 rather than from a separate offload device 130. The virtualoffload device 130 provides the same functionality as described inassociation with FIGS. 1 through 5. In this manner the virtualcomponents 140 are logically partitioned to a single logical groupingwhich can consolidate the organization and optimization of the virtualcomponents 140.

FIG. 8 illustrates another embodiment of a virtual computing networkcomprising a plurality of physical computing devices 200. The physicalcomputing devices 200 are in communication with a physical offloaddevice 130 via a interconnect interface 206. In this embodiment, thephysical computing device 100 s are configured to share a commoninterconnect interface 206 that allows each of the physical computingdevice 100 s to be in physical and electrical communication with theoffload device 130. The interconnect interface 206 provides each of thephysical computing devices 200 with a high speed, high bandwidth, andlow latency connection to the offload device 130.

The offload device 130 can have a plurality of virtual offload devices230 with each virtual offload device 230 associated with a specificphysical computing device 200. In this embodiment, there are threephysical computing devices 200 and three virtual offload devices 230 onthe hardware offload device 130. The virtual offload devices 230 work inthe same manner as the offload device 130 described in association withFIGS. 1 through 5. The offload device 130 can be partitioned between thethree different devices and so there can be a partition of the deviceassociated with each physical computing device 200.

It is to be understood that not necessarily all objects or advantagesmay be achieved in accordance with any particular embodiment describedherein. Thus, for example, those skilled in the art will recognize thatcertain embodiments may be configured to operate in a manner thatachieves or optimizes one advantage or group of advantages as taughtherein without necessarily achieving other objects or advantages as maybe taught or suggested herein.

All of the processes described herein may be embodied in, and fullyautomated via, software code modules executed by a computing system thatincludes one or more general purpose computers or processors. The codemodules may be stored in any type of non-transitory computer-readablemedium or other computer storage device. Some or all the methods mayalternatively be embodied in specialized computer hardware. In addition,the components referred to herein may be implemented in hardware,software, firmware or a combination thereof.

Many other variations than those described herein will be apparent fromthis disclosure. For example, depending on the embodiment, certain acts,events, or functions of any of the algorithms described herein can beperformed in a different sequence, can be added, merged, or left outaltogether (e.g., not all described acts or events are necessary for thepractice of the algorithms). Moreover, in certain embodiments, acts orevents can be performed concurrently, e.g., through multi-threadedprocessing, interrupt processing, or multiple processors or processorcores or on other parallel architectures, rather than sequentially. Inaddition, different tasks or processes can be performed by differentmachines and/or computing systems that can function together.

The various illustrative logical blocks, modules, and algorithm elementsdescribed in connection with the embodiments disclosed herein can beimplemented as electronic hardware, computer software, or combinationsof both. To clearly illustrate this interchangeability of hardware andsoftware, various illustrative components, blocks, modules, and elementshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular application and design constraints imposedon the overall system. The described functionality can be implemented invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the disclosure.

The various illustrative logical blocks and modules described inconnection with the embodiments disclosed herein can be implemented orperformed by a machine, such as a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general purpose processor can be a microprocessor,but in the alternative, the processor can be a controller,microcontroller, or state machine, combinations of the same, or thelike. A processor can include electrical circuitry configured to processcomputer-executable instructions. In another embodiment, a processorincludes an FPGA or other programmable device that performs logicoperations without processing computer-executable instructions. Aprocessor can also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. Although described hereinprimarily with respect to digital technology, a processor may alsoinclude primarily analog components. For example, some or the entiresignal processing algorithms described herein may be implemented inanalog circuitry or mixed analog and digital circuitry. A computingenvironment can include any type of computer system, including, but notlimited to, a computer system based on a microprocessor, a mainframecomputer, a digital signal processor, a portable computing device, adevice controller, or a computational engine within an appliance, toname a few.

The elements of a method, process, or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module stored in one or more memory devices andexecuted by one or more processors, or in a combination of the two. Asoftware module can reside in RAM memory, flash memory, ROM memory,EPROM memory, EEPROM memory, registers, hard disk, a removable disk, aCD ROM, or any other form of non-transitory computer-readable storagemedium, media, or physical computer storage known in the art. An examplestorage medium can be coupled to the processor such that the processorcan read information from, and write information to, the storage medium.In the alternative, the storage medium can be integral to the processor.The storage medium can be volatile or nonvolatile. The processor and thestorage medium can reside in an ASIC. The ASIC can reside in a userterminal. In the alternative, the processor and the storage medium canreside as discrete components in a user terminal.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are otherwise understoodwithin the context as used in general to convey that certain embodimentsinclude, while other embodiments do not include, certain features,elements and/or steps. Thus, such conditional language is not generallyintended to imply that features, elements and/or steps are in any wayrequired for one or more embodiments or that one or more embodimentsnecessarily include logic for deciding, with or without user input orprompting, whether these features, elements and/or steps are included orare to be performed in any particular embodiment.

Disjunctive language such as the phrase “at least one of X, Y, or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y, or Z, or any combination thereof (e.g., X, Y, and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y, or at least one of Z to each be present.

Any process descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or elements in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown, or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved as would be understood by those skilled in the art.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1. A computing system comprising: a first physical computing devicecomprising one or more processors and memory, wherein the one or moreprocessors are configured to execute instructions that, upon execution,configure the physical computing device to host a first plurality ofvirtual machine instances; a second physical computing device comprisingone or more processors and memory, wherein the one or more processorsare configured to execute instructions that, upon execution, configurethe physical computing device to host a second plurality of virtualmachine instances; an offload device, wherein the offload device is incommunication with the first physical computing device and the secondphysical computing device via an interconnect interface, the offloaddevice comprising one or more processors and memory, wherein the one ormore processors are configured to execute instructions that, uponexecution, configure the offload device to: instantiate a first virtualoffload device configured to host a first plurality of virtualinput/output (I/O) components associated with the first plurality ofvirtual machine instances on the first physical computing device,wherein the first plurality of virtual I/O components are configured toperform I/O functions on behalf of the first plurality of virtualmachine instances; and instantiate a second virtual offload deviceconfigured to host a second plurality of virtual I/O componentsassociated with the second plurality of virtual machine instances on thesecond physical computing device, wherein the second plurality ofvirtual I/O components are configured to perform I/O functions on behalfof the second plurality of virtual machine instances.
 2. The system ofclaim 1, wherein offload device is configured to instantiate the firstplurality of the virtual I/O components on the first virtual offloaddevice based, at least in part, on instructions received from a controlplane manager.
 3. The system of claim 2, wherein the virtual machinemanager is configured to provide addressing information associated withthe plurality of virtual I/O components to the plurality of virtualmachine instances so that at least some of the I/O requests generated bythe virtual machine instances are performed by the virtual I/Ocomponents.
 4. The system of claim 1, wherein the first virtual offloaddevice is partitioned from the second virtual offload device.
 5. Thesystem of claim 1, wherein the interconnect interface is a PeripheralComponent Interconnect Express interface.
 6. A computer-implementedmethod comprising: instantiating, by an offload device, a first virtualoffload device, wherein the first virtual offload device is associatedwith a first physical computing device, wherein the offload device is incommunication with the first physical computing device via aninterconnect interface; instantiating, by the offload device, a secondvirtual offload device, wherein the second virtual offload device isassociated with a second physical computing device, wherein in theoffload device is in communication with the second physical computingdevice via the interconnect interface; hosting, by the offload device, afirst plurality of virtual input/output (I/O) components on the firstvirtual offload device, wherein the first plurality of virtual I/Ocomponents are associated with at least one virtual machine instanceinstantiated on the first physical computing device, wherein theplurality of virtual I/O components are instantiated using the hardwarecomputing resources of the offload device; and hosting a secondplurality of virtual I/O components on the second virtual offloaddevice, wherein the second plurality of virtual I/O components areassociated with at least one virtual machine instance instantiated onthe second physical computing device, wherein the plurality of virtualI/O components are instantiated using the hardware computing resourcesof the offload device.
 7. The method of claim 6, wherein the firstvirtual offload device operates independently of the second virtualoffload device.
 8. The method of claim 6, wherein the first virtualoffload device is assigned a first range of memory addresses that isdifferent from a second range of memory addresses assigned to the secondvirtual offload device.
 9. The method of claim 8, wherein the firstplurality of virtual I/O components are assigned addressing information,wherein the addressing information comprises addresses in the firstrange of memory addresses.
 10. The method of claim 9 further comprisingresolving, by at least one of the first plurality of virtual I/Ocomponents, I/O requests generated by the first plurality of virtualmachine instance.
 11. The method of claim 6, wherein the first virtualoffload device is isolated from the operation of the second virtualoffload device.
 12. The method of claim 6, wherein the interconnectinterface is a Peripheral Component Interconnect Express interface. 13.An offload device comprising: one or more processors, wherein theoffload device is configured to operably couple to an interconnectinterface for communication with a first physical computing device and asecond physical computing device, the one or more processors areconfigured to execute instructions that, upon execution, configure theoffload device to: instantiate a first virtual offload device associatedwith the first physical computing device; host a first plurality ofvirtual input/output (I/O) components on the first virtual offloaddevice, wherein the first plurality of virtual I/O components configuredto emulate one or more physical I/O components using hardware computingresources of the offload device, wherein the first plurality of virtualI/O components are associated with at least one virtual machine instanceon the first physical computing device; and instantiate a second virtualoffload device associated with the second physical computing device;host a second plurality of virtual I/O components on the second virtualoffload device, wherein the second plurality of virtual I/O componentsconfigured to emulate one or more physical I/O components using hardwarecomputing resources of the offload device, wherein the second pluralityof virtual I/O components are associated with at least one virtualmachine instance on the second physical computing device.
 14. Theoffload device of claim 13, wherein the first plurality of virtual I/Ocomponents are assigned addressing information such that at least someof the I/O requests generated by the virtual machine instances of thefirst physical computing device are performed by the first plurality ofvirtual I/O components.
 15. The offload device of claim 13, wherein thefirst virtual offload device is assigned a first range of memoryaddresses that is different from a second range of memory addressesassigned to the second virtual offload device.
 16. The offload device ofclaim 15, wherein addressing information for the first plurality ofvirtual I/O components comprises memory addresses in the first range ofaddresses.
 17. The offload device of claim 13, wherein the first virtualoffload device is assigned virtual processing resources and virtualmemory resources corresponding to a first allocation of computingresources of the offload device and the second virtual offload device isassigned virtual processing resources and virtual memory resourcescorresponding to a second allocation of computing resources of theoffload device.
 18. The offload device of claim 13, wherein the firstplurality of virtual I/O components are instantiated in the virtualmemory resources of the first virtual offload device.
 19. The offloaddevice of claim 13, wherein the interconnect interface is a PeripheralComponent Interconnect Express interface.
 20. The offload device ofclaim 14, the first and second plurality of virtual I/O componentscomprise at least one of are at least one of a storage controller,network interface controller, or a programmable interrupt controller.